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Abstract 

In this article we shall survey the various methods of evaluating Hankel determinants and 
as an illustration we evaluate some Hankel determinants of a g-analogue of Catalan num- 
bers. Here we consider ^bq^q) as a g-analogue of Catalan numbers C n = ^qrr( 2 ? "), which is 
known as the moments of the little g-Jacobi polynomials. We also give several proofs of this 
q-analogue, in which we use lattice paths, the orthogonal polynomials, or the basic hypergeo- 
metric series. We also consider a g-analogue of Schroder Hankel determinants, and give a new 
proof of Moztkin Hankel determinants using an addition formula for 2-F1. 



1 Introduction 
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Given a sequence ao, 01, a2,. . . , we set the Hankel matrix of the sequence to be 

= (ffli+j+t)o<i,j<n-l 

For n = 0, 1,2, ... , let 
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which are called the Catalan numbers. The generating function for the Catalan numbers is 
given by 



]T c„t n = 



1 - yi - At 

2t 



If we put a n = C' n in ()1.1|) . then the following identity is well-known and several proofs are 
known El H3 E2]: 



det A$ = det (Gh-j+Oo^^^.! = II 



l<i<j<t-l 



i+j + 2n 
i + j 



If we put B n 



and D n = f ), then the following variations are also known |17| : 



det (B i+j+t ) 
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<i,3<n-l- 11 i+j -I 

l<i<j<t-l J 



det (Di+j+t) 



0<i,j<n-l 



= 2" 
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l<i<j< 
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As a generalization of (|1.3I) . Krattenthaler [TJ] has obtained 



det (C7fc i+j ) < <ij < n _ 1 = (fcj - fc<) JT 

0<i<j<n-l !=0 



(i + n)!(2fcQ! 

(2iy.ki\(h + ny. 



(1.6) 



for a positive integer n and non-negative integers fco, fci,. . . , k n -i. 

In this article we shall survey the various methods of evaluating Hankel determinants and 
as an illustration we give a q-analogue of the above results. We first recall some terminology 
in q-series (see Gasper-Rahman's book [9]) before stating the main theorem. Next some 
terminology is defined before stating the main theorem. We use the notation: 

OO 71 — 1 

(a;g)oo = Y\(l - aq k ), (a; q) n = JT (1 - aq k ) 

fc=0 fc=0 

for a nonnegative integer n > 0. Usually (a;q) n is called the q-shifted factorial, and we 
frequently use the compact notation: 

(ai, a2, . . • , a r ; q)^ = (oi; q) 00 {a2\ g)oo ■ ■ ■ (a r ; g)oo, 

(oi, ti2, . . • , a r ; g)„ = (oi; g)n(aa; g)n ■ • ■ (a r ; g)n- 

If we put a = q a and g — ► 1, then we have 

n-1 

where (a)n = ff ( Q + ^) i s called the raising factorial. We shall define the r +i4>r basic 

fc=0 

hypergeometric series by 



£ll, £12, . . . , flr+1 

h , . . . , b r 



= E 



(ai 

(g, 6i, . . . ,br;g)n 



If we put cii = g Qi and 6j = g ' in the above series and let g — >• 1, then we obtain the r +iF r 
hypergeometric series 



-lF r 



Ql, Ct2, ■ ■ ■ , Qr+1 



/3i,---,/3. ' 
The Motzkin number M n is defined to be 

M n = 2 Fi 



= E 



(ai)n(a2) n • • • (Or+l) 
n!(/3i)„...(^ r )„ * ' 



(1 - n)/2, -n/2 



;4 



The generating function for the Motzkin numbers is given by 



„ 1 — x — Vl — 2a; — 3a; 2 



2x 2 



det (Mi+i)^^ 



<i,j<n—l 



l 



It is known [I] that 
for n > 1, and 

det (Mi+i+iJo^j^.! = 1,0,-1 
for n = 0, 1 (mod6), n = 2, 5 (mod6), n = 3, 4 (mod6), respectively. 
The large Schroder number S n is defined to be 

5, = 2 2 F 1 [-" + 1 2 " + 2 ; -l 

for n > 1 (So = 1). The generating function for the large Schroder numbers is 

CO 

Eu and Fu [8] have proved 



(1.7) 
(1.8) 



1 — a; — Vl — 6a; + a; 2 
2~r 



det (5»4 



det (S, 



*+J + lio<i,j<n-l 



*+V0<i,.j<«-l 

for n > 1 (see [HO [16]). We can also prove that 

det(S i+i+2 ) < ij .< n _ 1 = 2("^)(2" +1 - 1) 

holds for n > 1. 



2CJ 1 ) 
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(1.10) 
(1.11) 
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In this article, as a generalization of H1.2|) . we choose 

(a&g 2 ;g) n 

for a nonnegative integer n. The aim of this article is to give three different proofs of the 
following theorem: 

Theorem 1.1. Let n be a positive integer. Then we have 

to^Wn-^-^ v ( 'n (a¥ -H 1;i , (a¥;} ) 2M) ' (i-") 

As a corollary of this theorem we can get the following more general identity. 
Corollary 1.2. Let n be a positive integer, and t a nonnegative integer. Then we have 

det (» i+j+t ) 0<i = a i«C- V«C»-i)(3«-i)+^-i>* / V* 



Proof. If we use 

fln+t = 



(a&g 2 ;g) 

.1 (aJ^-fc+t+l; q) n _ fc (aV+ 2 ; g) 2(n ~ 
(ag;g)„ +t (ag;g)t (aq t+1 ;q) 



y TT (g,ag t+1 ,feg;g) n - fc 



(abq 2 ;q) n+t (abq 2 ;q) t (abq t+2 ;q) r 
then we have 

^ ^ = det r ■ fou g fo 



V ( a6 ? 2 ; ?)< (abg'+ 2 ; g) 1 +i / <i, 3 <™-i 
= [ (ag;g)t ]" dct f (« , g;g).+j N 

\ (abg 2 ; g) t J V (a'&g 2 ; g);+j / < ! ,,<„-i 



where a' = ag J . If we use H1.13[) . then we obtain l|1.14|) by a straightforward computation. □ 

We can prove (|1.3|) , (|1.4[) and (|1.5|) as a corollary of Corollary 11.21 
Proof of (|l,3[l . (|1.4[l and (|1.5[1 . If we substitute a — q a and 6 = g' 3 into i/ n , and we put 

(q+/3+2)„ ' 



g — > 1, then we obtain fi n — > , ^TtV,") i which we write v n . Thus (|1.14[l . leads to 



, , , s _ n 1 T - fc)!(Q + t+ l)n-k(P + l)n-fc 

det ^ +J+t j <. - */ t JUL ( Q + £ + t + „ _ fc + i)„_ fc ( Q + /3 + t + 2 ) 2(n _ fc) 
Note that 



'C n /2 2n if a = -1 and /3 = ±, 



-,2n 

- -J a, iiU M - 2' 

B„/2 2 ' 1 if a = 1 and /3 = -§, 
D n /2 2n if a = -l and/3 = -i 



Hence we obtain 



{det (Ci+j+t)^ if a = -i and /3 = |, 
det (B l+J +Oo< l , 3 <n-i if Q = | and /3 = -§, 
det (D i+J+t ) < ij .< n _ 1 if a = -i and /3 = -§. 

Thus we can prove H1.3|) . p.4p and |T3J by direct computations from the above identity. □ 
In fact we can also obtain the following generalization of (|1.6[) . 



Theorem 1.3. Let n be a positive integer, and ko, ■ ■ ■ , k n -i nonnegative integers. Then we 
have 



(1.15) 
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(0,0) x 

Figure 1: A Dyck Path starting from (0, 0) and ending at (8, 2) 



2 Non-intersecting lattice paths 

In this section we give our first proof of Theorem 1 1 . 1 1 using non-intersecting lattice paths. 

Let m and n be nonnegative integers. A Dyck path is, by definition, a lattice path in the 
plane lattice Z 2 consisting of two types of steps: rise vector (1, 1) and fall vector (1, —1), which 
never passes below the rr-axis. We say a rise vector (resp. fall vector) whose origin is (x,y) 
and ends at (x + l.y + 1) (resp. (x + 1, y — 1)) has height y. For example, Figure [1] presents a 
Dyck path starting from (0, 0) and ending at (8, 2), in which each red number stands for the 
height of the step. Let & m ,n denote the set of Dyck paths starting from (0, 0) and ending at 
(m,n). Especially, the cardinality of £>2n,o is known to be the Catalan number C n - 

A Motzkin path is, by definition, a lattice path in Z 2 consisting of three types of steps: 
rise vectors (1, 1), fall vectors (1, —1), and (short) level vectors (1, 0) which never passes below 
the :r-axis. We say a rise vector, fall vector and level vector whose origin is (x, y) and ends 
at (x + l.y + 1), (x + 1, y — 1) and (x + 1, y) has height y, respectively. Figure [2] presents a 
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~\3 ' ' " 


• - ' - yf— ** ' 


(9,2) 







(0,0) x 

Figure 2: A Moztkin path starting from (0,0) and ending at (9,2) 

Motzkin path starting from (0,0) and ending at (9,2), in which each red number stands for 
the height of the step. Let ^£ m ,n denote the set of Motzkin paths starting from (0, 0) and 
ending at (m,n). Note that the cardinality of ^ n ,o is known to be the Motzkin number M„. 
We define the height of each step similarly as before. 

A Schroder path is, by definition, a lattice path in Z 2 consisting of three types of steps: 
rise vectors (1,1), fall vectors (1,-1), and long level vectors (2,0) which never passes below 
the x-axis. Figure [3] presents a Schroder path starting from (0,0) and ending at (10,0), in 
which each red number stands for the height of the step. Let S^ m ,n denote the set of Schroder 
paths starting from (0,0) and ending at (m,n). Note that the cardinality of ^n.o is known 
to be the large Schroder number S n - 

Assign the weight a^, fe/ M c^, to each rise vector, fall vector, (short or long) level vector of 
height h, respectively. Set the weight of a path P to be the product of the weights of its edges 
and denote it by w(P). Given any family & of paths, we write the generating function of & 
as 

GF [J?] = w ( p )- 

Proposition 2.1. (Flajolet [7]) The generating function for the Dyck paths is given by the 
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(0,0) 



(10,0) x 



Figure 3: A Schroder path starting from (0,0) and ending at (10,0) 



following Stieltjes type continued fraction: 



J2 GF [%„, 0) ] i 



2n 



1 



1 - 



qp bi i 2 



n>0 



1- 



Meanwhile, the generating function for the Motzkin paths is given by the following Jacobi 
type continued fraction: 



It is also easy to see the following proposition holds. 

Proposition 2.2. Let n be a positive integer. Then the generating function for Schroder 
paths is given by the following continued fraction: 



Next we recall notation and definitions used for the lattice path method due to Gessel 
and Viennot [10]. Let D = (V,E) be an acyclic digraph without multiple edges. If u and v 
are any pair of vertices, let (u, v) denote the set of all directed paths from u to v. For a 
fixed positive integer n, an n- vertex is an n-tuple of vertices of D. If u = (ui, . . . , u„) and 
v — (vi, . ..,«„) are n- vertices, an n-path from u to v is an n-tuple P = (Pi, . . . , P n ) such 
that Pi £ 3 s (ui,Vi), i = 1, . . . ,n. The n-path P — (Pi, . . . , P n ) is said to be non-intersecting 
if any two different paths Pi and Pj have no vertex in common. We will write (it, v) for 
the set of all n-paths from u to v, and write (u, v) for the subset of (u, v) consisting of 
non-intersecting n-paths. If u = (ui, . . . , u m ) and v — (vi, . . . , v n ) are linearly ordered sets of 
vertices of D, then u is said to be D-compatible with v if every path P £ {P(v,i,vi) intersects 
with every path Q £ ^(iij, Vk) whenever i < j and k < I. Let S„ denote the symmetric group 
on {1,2, . . . ,n}. Then for n £ S n , by v n we mean the n vertex (t)^-(i), . . . ,iV(n))- 

The weight w(P) of an n-path P is defined to be the product of the weights of its compo- 
nents. Thus, if u = (ui, . . . , u n ) and v = (vi, . . . , v n ) are n-vertices, we define the generating 
functions F{u,v) = G¥ (u,v)} = Y,p e &(*,,v) w ( P ) and ^o(«,«) = GF [&> (u, «)] = 
X^P6 i 5» ( li „) W (P)- l n particular, if it and « are any pair of vertices, we write 



The following lemma is called the Gessel- Viennot formula for counting lattice paths in terms 
of determinants. (See |10j.) 





h(u, v) = GF (u, v)] = M ( p )- 



Pe5*(u,u) 
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Lemma 2.3. (Lidstrom-Gessel-Viennot) 

Let u = (tti ,...,«„) and v = (vi, . . . , v„) be two n- vertices in an acyclic digraph D. Then 



^ sgn7r F (u 7T ,v) = det[/i(ui,Uj)]i<ij< n . 

vr6S„ 

In particular, if u is D-compatible with v, then 

P (u,i>) = det[/i(wi,%-)]i<«j'<n- 
If we apply Lemma 12.31 to Dyck paths, then we obtain the following proposition: 

Proposition 2.4. Let G m = GF [$> 2m ,o] f° r non-negative integer m. 

(i) If t = 0, then we have 

n 

det (Gi+j) < j ^< n _ 1 = Y\ ( a 2i-2&2i-ia2i-i&2i) n_I • 

i=l 

(ii) If t = 1, then we have 

det (G i+J+1 ) <. = (a 2l _ 2 6 2 ^ 1 )- l+1 (a*-^*)""' . 

!=1 

(iii) If t — 2, then we have det (Gi+j+ 2 ) 0<i j< n _ 1 equals 

n k k 

II ( a ° ai ' ' ' a 2i-3 a 2i-2bib 2 ■ ■ ■ &2i-l&2i-l) ' f7 ( a oai • • • a 2i -ib 1 b 2 ■ ■ ■ b 2i ) . 

fc=0i=l i=l 

(iv) If t = 3, then we have det (Gi+j+3) 0<i 3 -<n-i e 1 ua l s 

n k l k 



(2.1) 



(2.2) 



(2.3) 



(2.4) 



(2.5) 



]"[ (apai • • • g2i-3a2i-2&2i-l) ] [ (aoai---02i-302i-202i-l&2i)| 
fe=0 i=0 i=l i=l+l 
k I k 

x |^ Y\ {bib 2 ■ ■ ■ b 2i - 2 bl i _ 1 a 2i - 2 ) ] [ (bib 2 ■ ■ ■ b 2 i- 2 b 2 i-xb 2l a 2i -x) | 



i=I + l 

x [ (aoai • ■ ■ a 2i _ia 2i 6i6 2 • ■ • 6 24 6 24+ i) . (2.6) 

i=k+l 

((i) and (ii) of this proposition are originally appeared in [181 Ch. 4, §3].) 

Proof. We consider the digraph (V,E), in which V is the plane lattice 1? and E the set 
of rise vectors and fall vectors in the above half plane. Let Ui = (xo — 2(i — 1),0) and 
Vj — (xo + 2(j + t — 1), 0) for i, j = 1, 2, . . . , n, t = 0, 1, 2, 3 and a fixed integer xq. It is easy 
to see that the n- vertex u = (ui, . . . , u n ) is D -compatible with the n- vertex v — (vi, . . . , v n ). 
If t = 0, then there is always a unique n-path P = (Pi, . . . , P n ) that connect u to v as in 




(0,0) u 4 u 3 u 2 ui=Vi V2 V3 V4 x 



Figure 4: t = and n = 4 

Figure U By multiplying the weights of all edges in P, we obtain the right-hand side of 
On the other hand, applying Lemma 12.31 we obtain the left-hand side of (|2.3I) . 

The other identities can be proven similarly. For example, if t = 1, there is only one n-path 
P = (Pi, . . . , P n ) that connect u to v as in Figure [S] As the product of the weights of all 
edges in P we obtain (|2.4|l . If t = 2, there are (n + 1) ways to connect u to v with n-path 
P = (Pi, . . . , P n ). As an example, we show one way in Figure [6] A similar reasoning leads to 
(|2.5p . One can also derive (|2.6p by a similar argument. □ 



G 




We assign the following weight to each step: the weight of a rise vector is 1, while the 
weight of a fall vector of height h is 



aq k (l-q k )(l-bq k ) -r , _ r,, . 

(l-a6 9 ^)(l-a6 9 ^ + i) " " " * 1S eVeI1 - 



„ fc + M 



(2.7) 



For example, we have Ai = j^ffr, A 2 = 0^%^) , As = (Lat^j(i-°4 J j ■ and an 
example of the weight of a path is Figure [Jj 

Lemma 2.5. Let m and n be a non-negative integers such that m = n (mod 2). Then the 
generating function of @ m ,n is given by 



GF (@ m , n ) = 



LfJ 
LfJ 



(aa 1+r t1; a ) m _„ 

2 

(a6g 2 +";(7)m- ii 



(2.8) 



Here [sj (resp. [x] ) stands for the greatest integer that does not exceed x (resp. the smallest 
integer that is not smaller than x). Especially, we have 



GF (& 2n ,o) 



("g;g)n 
(abq 2 ;q) n 



(2.9) 



Proof. We prove (|2.8[) by induction on m. If m = 0, then it is obvious that GF (So,n) equals 
1 if n = 0, and otherwise. Assume that (|2.8|) holds up to m — 1. Then we have 



GF [JU,n] = GF [9m-l,n-l] + A n+1 GF [» m -l,n+l] • 

If m = 2r and n = 2s, then, by induction hyperthesis and the above recursion, we obtain 




(0,0) "4 "3 "2 "1 



^1 ^2 ^3 V A X 



Figure 6: t = 1 and rt = 4 
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Figure 7: A Dyck Path of weight AiA 2 , 



GF [&2r,2s] equals 



r - 1 
s - 1 



(aq° +1 ;q) 



+ 



g s (l-ag fl+1 )(l 



(abq 2s + 1 ;q) r _ s (1 - abq 2 ^ 1 )^ - abq 2s + 2 ) 



r - 1 
s 



« +2 ;' ? ),- s - 1 

(a6g2=+3;g) r _ s _ 1 



;9) 7 ._ 1 « +1 ;?) r 



(<?;</)„ (<7;<?) r _ s (a6g 2s+1 ;<?) 



{(1 - g s ) (1 - a&<f + g s (l - <f-) (1 - abg s+1 )} 



1 — s+1 



rr i 



LsJ, (abq 2s + 2 ;q) r _ s ' 

This equals the right-hand side of (|2.8|) with m = 2r and n = 2s. Hence 1)2. 8|) holds when 



GF [0 m _i,„+i] 



GF [0 m ,„] 



GF [9 m -l,n-l} 



Figure 8: GF 1 = GF i] + A„+iGF [2i m -i,n+i\ 



m — 2r. One can prove (|2.8p similarly when m = 2r + 1 and n = 2s + 1. □ 
For example, if m — 4 and n = 0, then ^4,0 has the two Dyck paths shown in Figure [9] Thus, 



(0,0) 



(4,0) 




(0,0) 



(4,0) 



Figure 9: Dyck Paths in S>4 t0 



the generating function of S^o equals 



GF(^,o) - A! + ArA 2 - (1 _ aV)(1 _ o6?3) - 



Proof of Theorem ll.il If we use (|2.3[) . (|2.7p and (|2.9[) . then we conclude that det (jJ.j+j) 0<i J<n x 
equals 



^(A^rA^r-^n 



n r „„2i-l 



ag 2 '-^! - g')(l - gg')(l - bg')(l - abq 1 ) 
(1 - afeg 2i_1 )(l - abg 2l ) 2 (l - afeg 2 ^ 1 ) 



<S 



An easy computation leads to (|1.13|) . □ 

Remark 2.6. One can also prove Theorem 1 1.1 1 by using Motzkin paths and giving the weight 
A2h+i to rise vector of hight h, \2h to fall vector of hight h and \2h + ^2h+i to level vector of 
hight h. Then one can prove 

GF (^ m , n ) = \ m } W-^ . (2.10) 
in\ q (a6g»+ 1 ;g) m+ i 

for nonnegative integers m and n. 



3 Orthogonal Polynomials 

In this section we give our second proof of Theorem 1 1 . 1 1 using the little g-Jacobi polynomials. 
We use the notation S(t; Ai, A2, . . .) for the Stieltjes-type continued fraction 

(3.1) 



Ait 



1 



A 2 t 



1 



and J(t; bo, bi, 62, . . . ; Ai, A2, . . . ) for the Jacobi-type continued fraction 

1 



1 — box — 



1 — b\X — 



(3.2) 



1 — b n x — 



X„x 



Given a moment sequence {^n}, we define the linear functional ,S£ : x n n> fi n on the vector 
space of polynomials C[x]. Then the monic polynomials p n (x) orthogonal with respect to Jzf 
and of degp n (a;) = n satisfy a three term recurrence relation (Favard's theorem), say 



p n+1 (x) = (x - b n )p„(x) - X„p n -i(x), 



(3.3) 



where p-i(x) = and po{x) = 1. The moment sequence {fin} is related to the coefficients b n 
and A n by the identity: 



1 + ^2 HnX n = J(t; b , 61,62, . . . ; Ai, A 2 



(3.4) 



Hereafter we assume Ao = fio — 1 for simplicity of arguments. 
Define A n and D n (x) by 



Mo Mi 
Mi M2 

Mn Mn+l 



/'') 

Mn + 1 
M2n 



D n (x) 



Mo Mi 
Mi M2 



f-Ln — 1 fJ'n 

1 x 



Then p n (x) = (A n _i) _1 _D n (a;) is the monic OPS for Jzf. 
It is easy to see that 



Mn+i 



M2n-1 

x n 



J?(x n p n (x)) = 



A„_i 



A„_i 



where 



A n An— 1 • • • A1M0, 
AnA„-i . . . Ai^to(&o 



Mo Mi 

Mi M2 

M"" 1 M" 

Mn+l Mn+2 



fin 

Mn+i 



M2n-1 
M2n+1 



(3.5) 
(3.6) 
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Therefore 



and 



An — 



■gjgjUg))] An- 2 A n 

■gM(>))] _ 

i^KW)] ~ AT ~ A„_i' 



(3.7) 



(3.8) 



Theorem 3.1 (The Stieltjes-Rogers addition formula). The formal power series f(x) 
^2 i>Q aiX l /i\ (do — 1) has the property that 

f(x + y)=^2 ccmf m (x)f m (y), 



where a m is independent of x and y and 



fm (X) — - + /?,„ 

m! (m + 1)! 



~,m+l 

+ 0(x m+2 ), 



if and only if the formal power series f(x) = X^i>o ai:rl nas ^ ne J-continued fraction expansion 
J(ie; bo, bi, 62, ■ • ■ ; Ai, A2, ■ ■ ■ ) with the parameters 



/3m+i — An an d An 



m > 0. 



From (|3.5p . one can compute the Hankel determinants 



dct (Mi+j)o<i,j<,i-i — A„_i — fig A" A2 ■ • ■ A„_ 2 A„ 



(3.9) 



of (|1.13[) by taking appropriate orthogonal polynomials p„(x). Recall the definition of Heine's 
g-hypergeometric series 



2(pi(a,b-c;q-x) = ^ 



(a;q) n (b;q) n x n 
(c;g)„ (q;q)r 



The following is one of Heine's three-term contiguous relations for 2</>i: 

, . 111 \ (1 — a )( c ~ b) , 2 \ 

2 cpi{a,b;c;q;x) = 291(0, oq; cq; q; x) + — — -x 2<pi{aq,bq; cq ;q;x). 



(l-c)(l-cq) 



2<t>i(aM; eg; q;x) 

20i (a, b; c;q;x) 

(l-a)(6-c) (l-6g)(a-eg) (1 - ag)(6g - eg 2 ) 



It follows that 



(1 - c)(l - cq) ' (1 - cq)(l - cq 2 ) ' (1 - cg 2 )(l - cq 3 ) 
Hence, by induction, we can prove that 

2<j>i{a,bq;cq;q;x 



where 



Aan-i 



20i (a, b;c;q;x) 
(1 - ag")(6 - cg")g" 



= S(x; Ai, A 2 , . . . ), 

(1 -&<?")(<! -eg")?"- 1 

A2n — 



(l-cg 2 ")(l-cg 2 ' l + 1 )' (l-cg 2 "- 1 )(l-cg 2 ") ' 

Making the substitution 6 <— 1, a <— aq and c afeg into the above equation, we obtain 

E(aq;q) n n . . 



where 



A2n + 1 — 



(abq 2 -q)„ 



(l-ag n+1 )(l-a&g n+1 )g ,, 



A2™ — 



(l-g«)(l- 6 g>g* 



(l-ag 2 "+ 1 )(l-afcg 2n+2 ) ' (l-o!ig 2 ")(l-fl!)g 2 "+ 1 )' 
This corresponds to the little g-Jacobi polynomials. Indeed, the little g-Jacobi polynomials 

'q- n ,abq n+1 



p n (x;a,b;q)= {abqn+1 _ q) J -l) g W 3 fc 



aq 



; i,xq 



(3.10) 
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are introduced in [2J. The polynomials satisfy the recurrence equation 

Xp„(x) = p n +l{x) + (A„ + C„)p„(x) + A n -lC n -lPn-l(x) (3-11) 

where p-i(x) — 0, Po(x) = 1 and 

_ g"(l~ag" + 1 )(l-a6 g "+ 1 ) ag"(l - g")(l - bg") 

" (l-a&g 2 "+ 1 )(l-abg 2 ' l + 2 )' (1 - abq 2 ")(l - abq 2 ^ 1 ) ' 1 ' 

They are orthogonal with respect to the moment sequence {^n} n>0 where 

= 7 g^. (3.13) 
(abq 2 ;q) n 

For the passage from the Stieltjes-type continued fraction to the Jacobi-type continued fraction 
we use the following contraction formula: 

S(x, Ai, A 2 , . . . ) = 



. , \1\2t 

Alt — 



(A 2 +A 3 )t-^ 



Thus, by the same computation as in the former section, we conclude that the determinant 
(13.131) is equal to 13f) . This proof gives us an insight to the determinant (|1.13p from the 
point of view of the classical orthogonal polynomial theory. 



4 g-Dougall's formula 

In this section we give our third proof of Theorem 11.11 using g-Dougall's formula and LU- 
decomposition of the Hankel matrix. 

First the following formula is known as g-Dougall's formula: We have 



a,qa,2 ,—qai ,b,c,d aq 
aa , -a! , aq/b,aq/c, aq/d' bed 



(qa, aq/bc, aq/bd, aq/cd; q) c 
(aq/b, aq/c, aq/d, aq/bed; q) 



provided \aq/bcd\ < 1 (see [9] (2.7.1)]). If we perform the substitution a <— abq, b 
c <— q~ l and d ^— g~ J in (|4,1[) . then we obtain 



abq,ah2q2,-aib2q2,bq,q \q 3 _ 

ill ill j 9 j @>C[ 

az b? qz , —a? bz qz , aq, abq l+2 , abq J+2 



where u n = Sl q .?\ as before. If we use 



(?;?)n 



(-!)*<? 



h „(2)- nk 



(q;q)n-k 

then this identity can be rewritten as 

(q,abq,aibiqi ,-gibiqi ,bq; q) k 



i 








1 


k 



q (a2&2 q2 , — a2&2 52 , aq, abq i+2 , abqi+ 2 ;q)k M>/Aj 



If we put 



= (M {abq J+2 ;q) j 
13 H (abq*+ 2 ;q) 3 



Uij = a l q l Hifij 



(q, abq, 02^2^2 , -a? b? q? ,bq; q)i 



1,1 1 



(a 2 &2 g2 : —a2.b2.qn , aq, abq 1+2 ,abqi 



+2. 



then (|4.3[) implies 



(4.1) 

- bq, 

(4.2) 



(4.3) 

(4.4) 
(4.5) 



^ ^ ^ik^kj — 



(4.6) 



11 



Note that L n — (hj) 0<i j< n _i I s a lower triangular matrix such that all main-diagonal entries 
are 1, and U n = (uij) 0<i J<n _ 1 is an upper-triangular matrix with diagonal entries 



i- 2 (q,abq,a 1 2b2q2,-a 1 2b2q2,bq;q)i 
uu — a q fj,f , , ± 11 i 



(a 2 & 2 q2 1 —aib? qi , aq, abq z+2 , abq l + 2 ; q)i 



a 9 r 1 u o \2 • ( 4 - 7 ) 

l-afcq (abq 2 -^)^ 



r 1 — afeg 2l+1 (q,aq,bq,abq;q) 
l-abq (abq 2 ;q) 2 2i 

Since (M<+j')o<i,j<n-i = ^if/„, det (Mi+jOo^i ,^<n-i is tne product of the diagonal entries, i.e., 

dst (^i+j) <i,j<n-l = II Uii - 
i=0 

Using ()4.70 . one can easily prove 1)1.130 by a direct computation. 

Remark 4.1. Wc should note that Corollary 1 1 . 2 1 can be proven by induction using the fol- 
lowing Desnanot-Jacobi adjoint matrix theorem: If M is an n x n matrix, then we have 

det M det M{'* = det Mf det M™ - det M\ det M" , (4.8) 

where MJJ' denotes the (n — r) x (n — r) submatrix obtained by removing rows ii, . . . ,i r 
and columns ji, ... ,j r from M. 

Corollary 11.21 can be also proven as a special case of Theorem 11.31 which will be proven in 
Section [5. II In fact, if one puts ki — i + t in 1 )1.150 . then he obtains (11.1411 . 

5 Miscellany 

5.1 A proof of Theorem 11.31 

In this subsection we give a proof of Theorem 11.31 Before we prove the formula, we need to 
cite a lemma from 111! 1121 . 



Lemma 5.1 (Krattenthaler [TT]). Let Xq, X n , A\, A„-i, and Si, B n -\ be 
indeterminates. Then there holds 



det 



U(Xi + Bi) 11 {Xt+Ai 

1 = 1 1=3 + 1 



= n pi-**) n i*i- A i)- 

0<i,j<n-l 0<i<j<n-l l<i<j<n-l 

(5.1) 



Proof of Theorem 11.31 Using 

_ (aq;q)r 



(abq 2 ;q) r 
we can write 



n-l 



det (Mfc < +j) <i,j<n-i = n U^ti-i det n (i - n 

«=0 V y ,+ \;=1 £=3+1 



0<i,j<n-l 



1=1 1=3 + 1 



0<i,3<n-l 



If we substitute X% — q k ' , B; = — aq ! and A; = —abq l+1 into (|5.10 . then we see that 



(afcg 2 ; g) fci+ „_i 

One can derive 1)1.150 easily by a direct computation. □ 
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5.2 An addition formula for 2 F± 



In this subsection we give a new proof of (|1.7|) using an addition formula for 2-F1 and LU- 
decomposition of Motzkin Hankel matrices. First, we shall prove the following identity. 

Lemma 5.2. If i and j are nonnegative integers, then we have 



E 



2 Fi 



fc — i + l k — i 
2 ' 2 
k + 2 



fc-j+i fc-j 
2 ' 2 

fc + 2 



2F1 



k \k 

Proof. Recall the quadratic transformation formula (see [9] (3.1.5)]) 
(l-z) a 2F 1 (a,b;2b;2z) = 2 F 1 (£,£±i ; 6+i; 



- l — i—j —i—j 
2 ' 2 

2 



v 2' 2 ' 2' (1-z) 2 
Applying (|5.3|l with a = k — i, b = k + 3/2 and 2 = 2 we obtain 

k -i, fc + 3/2 " 



2 -Pi 

Substituting z by j yields 
2F1 



fc — i + l fc — i 
2 ' 2 

fc + 2 



fc-j+i fc-j 
2 ' 2 
fc + 2 



(-1)^2*1 



(-l) fe ^ 2 Fi 



2fc + 3 



fc-j, fc + 3/2 



2fc + 3 

Now, applying (|5.3|l with a = — i — j, b = 3/2 and 2 = 2 we obtain 



: 4 



iFi 



- 1 — i—j — i—j 
2 ' 2 
2 



: 4 



Therefore we can rewrite (15.21) as follows 



E 



M / J 
fc/ \fc 



k-i, fc + 3/2 



2.F1 



(-ly+^Fr 



fc - j, fc + 3/2 
2fc + 3 



-» - i- 1 



: 4 



2fc + 3 

Now we recall a formula of Burchnall and Chaundy [3] (43)] 
c — a. c — b 

■ x = 

c 

J fc>0 



1F1 



-i-3 



' 2 :4 



2*1 



= E 



(c - a) k (a)k{d)k(c - b - d) u ^ 2 k 



x 2*1 



fc!(c + fc- l) fe (c) 2 fc 
c — a + fc,c — & — d + fc 



: K 



c + 2fc 

It is then easy to check that the specialization of (|5.5[) with 



2*1 



c — a + k, d + k 
c + 2fc 



(5.2) 



(5.3) 



(5.4) 



(5.5) 



a= 2' & = 3 + i+j, c = 3, 



-j, x = 4 



yields (|5.4j) . 

Proof of (|1.7|l . Define and Uy by 



>*i 



!*i 



j — i+l j — i 

2 ' 2 . 4 

1 + 2 ' 

■ i—j+l i—j 

2 '2 . l 
i + 2 



□ 

(5.6) 
(5.7) 



Then L n = (Zi,-) 0<i ■<»— 1 ^ s a l° wer triangular matrix with all diagonal entries 1, and U n = 
(Uj) 0<i j<n— 1 ^ s an u PP er triangular matrix with all diagonal entries 1. The formula (|5.2[1 
gives T;he LU-decomposition of Motzkin Hankel matrix: 



Hence we conclude that det (Afjj) = 1. 



□ 
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5.3 A g-analogue of Schroder numbers 

We define S n (q) (n > 0) by the following recurrence: 

n-1 

S (q) = 1, S n (q) = q^Sn-^q) + £ gW^" 1 -^ (g)S fc (g). 

k-Q 

In fact one can show that 

S n {q)= £ 

where cj(P) is the number of triangles below the path P (see Figure [T0)l . and the sum runs 




(0,0) ^ *t- *(2n,0) 

Figure 10: Weight of Schroder paths in oS^o 

over all Schroder paths from the origin to (2n,0). As a g-analogue of (|l,10|l and we 
consider the matrix 

sP(q)= U'-^-^s^Mq)) ■ (5-8) 

V '0<ij'<n— 1 

Note that this matrix is not a Hankel matrix, but as a (/-analogue of (|l,l(jp and p. lip , the 
following theorem holds: 
Theorem 5.3. Let n be a positive integer, 
(i) If t — or 1, then we have 



n-1 



detS^g) = detS ( n 0) (q) = + l)""* (5.9) 

fe=i 

(ii) If t = 2, then we have 

n n + 1 

det5i 2) (g) = a" 1 Uiq 2 "- 1 + ir+^HKq 2 "- 1 + 1) - 1). (5.10) 

k=l fe = l 

To prove this theorem, we define the matrices 

V / l<i,j<n 

\ / l<i,j<n 

then the following lemma can be easily proven by direct computations: 
Lemma 5.4. Let n be a positive integer. Then 

detS™(q) = 1 3 detS£>(q), (5.11) 

~ n(4.i 2 +3(2t + l)Ti + 3t 2 +3t-l) ,,, 

detS 7l (q) = q 3 det S% } {q). (5.12) 

Lemma 5.5. Let n be a positive integer. 

(i) If n > 2, then we have 

detSW(g) = g^X™- 2 ' detS^fo). (5.13) 

(ii) If n > 2, then we have 

detS«(g) = g" 2 - +1 det ^.(g) + g 2( "- 1)2 detS^g). (5.14) 

(iii) If n > 3, then we have 

detSi 0) (g)det5^ 2 (g) = det S<l(g) det^?) - {detS^g)} 2 . (5.15) 
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Proof. We consider the digraph (V, E), in which V is the plane lattice Z 2 and E the set of rise 
vectors, fall vectors and long level vectors in the above half plane. Let Ui = (xo — 2(i — 1), 0) and 
Vj — (xo+2(j+t — 1), 0) for i, j = 1,2, . . . , n, t = 0, 1, 2 and a fixed integer xo- It is easy to see 
that the n- vertex u = (ui, . . . , u n ) is D-compatible with the n- vertex = (v^, . . . , v„ 
We assign the weight of each edge as a rise vector, a fall vector and a long level vector whose 




Figure 11: Weight of each edge 



origin is (x, y) and ends at (x+l,y + l), (x + l,y — 1) and (x + 2,y) has weight q x -v-^o+2{n-i) ^ 
1 and q x ~y~ x o+ 2n ~ 1 ) respectively, which is visualized in Figure ITT1 Then, by applying Lemma 
2.3, we can obtain 

GF [> = detSi 4) (g). (5.16) 

This is important to prove the following. 

(i) Assume t = and let u and w be as above. Put vn = (xo — 2i + 3, 1) and = (xo + 2j — 3, 1) 
for i,j = 2, ...,n, and let u = («2, ...,U n ) and = («2 > • ■ ■ i Vn^)- Then each n-path 
P = (Pi, P2, . . . , P n ) from u to corresponds to an (n — l)-path P = (P2, . . . , P n ) from u 
to v 1 - 1 * 1 by regarding P as the subpath of P. In fact, note that Pi is always the path composed 
of a single vertex ui = Vi, each P; always starts from the rise vector m — > Ui and ends at 
the fall vector vj 1 ' — > v!> ' for i — 2,...,n. Hence this gives a bijection, and the product 
of the weight of the rise vectors Ui — > Ui and the fall vectors 55^ — > for i = 2, . . . , n is 

g (n-l)(n-2) TMg proveg (J57J3J, 

(ii) Assume t = 1 and let it and v be as above, i.e., Ui = (xo — 2(i — 1),0) and = 
(xo + 2j,0) for 1 < i, j < n (see Figure [12]). Put u t = (xo - 2i + 3, 1) (2 < i < n) and 




Figure 12: i = 1 case 



vf ] = (x + 2j - 1,1) (2 < j < n), and let u = (u 2 , . . . ,u n ) and v {2) = (^ 2) , . . . ,«£ 2) ). 
Further, put in = (x - 2i + 4, 2) (2 < i < n) and uj 1 ' = (x + 2j - 2, 2) (2 < j < n), and let 

■u = (H2, ■ ■ ■ ,u„) and i)^ = (fig, ■ • ■ , )■ Let P = (Pi,P 2 , . . . ,P n ) be any non-intersecting 
n-paths from u to Then, it is easy to see that P must satisfy one of the following two 

conditions: 

(1) Pi is the long level vector whose origin is tii and ends at v[ , and Pi goes through the 
vertices Ui and vf' for i = 2, 3, . . . , n. 
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(2) Pi is a path which goes through only three vertices ui, ui + (l,l)(= v[ 1] - (1,-1)) and 
v[ , and Pi goes through the vertices u», in, and u£ 2 ' for i = 2, 3, . . . , n. 
By a similar argument as in the proof of (i), we can deduce that 

GF [<? (u n ,vV)] =g" 2 -" +1 GF [^(ttn-i.w^)] +g 2( "- 1)2 GF [>„(Uft-l, 1#2i)] 

holds. By the equality (|5.16[) . we obtain the identity (|5.14|l . 

(iii) This identity can be proven by applying the Desnanot-Jacobi adjoint matrix theorem (|4.8[) 
to Sk t} (q). □ 

Proof of Theorem 15.31 (i) The first equality of (|5.9|) is easily obtained from (|5.11fl and 
(|5.13|) . By applying the equalities (|5. 1 If) and (|5.12|l to (|5.14|l and (|5.15j) . we have 

detSgHq) = qdetS^q) + dets£l 1 (q) (5.17) 

for n > 2, and we have 

det S^(q) det S^l 2 (q) = det S^M det S^q) - g^^fdet S^q)} 2 (5.18) 
for n > 3. By the equalities (|5.17[) and (|5.18p . for n > 3, the following identity holds: 
det S£\q) (det S^Uq) ~ detS« 2 (g)) 

= detS^ 1 (g)(detSi 1) ( ? ) -detS« x (g)) - g^fdet ^(g)} 2 . (5.19) 
Moreover, by applying the first equality of (|5.9I) to (|5.19|) and replacing n with n—1, we obtain 

(1 + g 2 - 3 ){det Si°\(g)} 2 = det S^ 2 (g) det S< 0) (g) (5.20) 



for n > 4. We prove the second equality of (|5.9[) by induction on n. If n = 1, 2, 3, then it is 
easily obtained by direct computations. Assume that (|5.9[) holds up to n— 1. Then, by (|5.20l) 
and induction hypothesis, we can obtain the second equality (|5.9|) . 

(ii) It follows from our result of (i) and the equality (|5.17l) that (|5.10|l holds. □ 
By applying the Desnanot-Jacobi adjoint matrix theorem 1)4. 8[l to SS^l), then we have 

det det 5^(1) = det Sg> (1) det (1) - {det si 2) (l)} 2 (5.21) 
for n > 2. Therefore the following identity is easily obtained by induction on n and the formula 

Remark 5.6. For positive integer n, we have 

det S^{1) = - {2n + 3)2('^ 2 ) - 2^) . (5.22) 

Note that there is a relation between domino tilings of the Aztec diamonds and Schroder 
paths (see [8] [15]). It might be an interesting problem to consider what this weight means. 

5.4 Delannoy numbers 

The Delannoy numbers D(a, b) are the number of lattice paths from (0, 0) to (b, a) in which 
only east (1,0), north (0,1), and northeast (1,1) steps are allowed. They are given by the 
recurrence relation 

D(a,0) = D(0,b) = 1, 

D(a, b) = D(a - 1, b) + D(a -1,6-1)+ D(a, b - 1). (5.23) 

The first few terms of D(n, n) (n — 0, 1, 2, . . . ) are given by 1, 3, 13, 63, 321, .... By a similar 
argument we can derive the following result. We may give a proof in another occasion. 

Proposition 5.7. Let n be a positive integers. Then the following identities would hold: 

det(D(i + j,i+j)) ^ n _ 1 = 2( n i 1 )-\ (5.24) 
det (£>(< + j + 1, i + j + 1)) < 1 , J <„_ 1 = 2("^H + 2( n * 1 )-\ (5.25) 
det (D(i +j + 2,i + j + 2)) < 4J < n _ 1 = 2("J 3 )" 3 + (2n + l^"^)" 2 - 2^ 1 )~ 1 . (5.26) 
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6 Concluding remarks 



Since a hyperpfaffian version of (|1.3[) is obtained in [13], we believe it will be interesting problem 
to consider a hyperpfaffian version of Theorem 11.11 and Theorem 11.31 We shall argue on it 
in another chance. The authors also would like to express their gratitude to the anonymous 
referee for his (her) constructive comments. 
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